1. Field of the Invention
The present invention relates, generally, to content delivery networks and, in preferred embodiments, to systems and methods for efficient redirection of content requests to mirror sites using http meta-tag protocols to improve content delivery services.
2. Description of the Related Art
Web performance is a key point of differentiation among content providers. Crashes and slowdowns within major web sites demonstrate the difficulties companies face in trying to deal with high web traffic. As Internet backbone technologies have developed, many innovations in the area of service management have improved bandwidth and web content retrieval response time. These improvements to infrastructure, however, cannot solve traffic problems at all points within the Internet.
For example, assume in FIG. 1 that an end user 12 in a network 14 in Japan requests access to a page from a web site 16 in a network 18 the United States. (For the purpose of simplifying the description, end users may be described herein as requesting or receiving content. However, it should be understood that the end user""s Web browser actually requests or receives the content.) The request must pass through several gateways 20, 88, and 90 before reaching web site 16. Although web site 16 may have large bandwidth (the ability to rapidly communicate large quantities of data), the gateways connecting the network 14 in Japan to the network 18 in the United States may be slow, and thus, when end user 12 attempts to access the page from web site 16, the gateways may create a bottleneck. Such gateway bottlenecks may result in the access time for one page of data being on the order of 10 seconds or more. Because of the gateway bottlenecks, and because there are many uncertainties along the Internet paths from end user 12 to/from web site 16, content delivery networks or systems are now being developed.
Fundamentally, content delivery systems are designed for two major purposes; one is to achieve load balancing, and the other is to reduce overall response time. A content delivery system may be implemented using a high speed dedicated line to deliver content while bypassing all gateways or reducing the number of Internet gateways in the transmission path. However, such a dedicated system is expensive. Another approach to implementing content delivery systems is through the use of intelligent caching, mirroring, proxy servers, or other techniques which redirect end users to available servers that contain copies of the requested content and that are close to (quickly accessible by) the end users. With some of the traffic redirected, traffic surges will decrease and end users will benefit from faster response time. The term generally used for the architecture and functionality of such networks or systems is content delivery services (CDS).
FIG. 2 is a block diagram illustrating an example of a conventional embedded object redirection content delivery system 10. In the example of FIG. 2, an end user 92 located within a local network 94, e.g., in Japan, requests access to a web page containing object 1 (obj1), object 2 (obj2), and object 3 (obj3) from a remote content provider original site 96 located within a network 98 in, e.g., the United States. Obj1, obj2, and obj3 may comprise, for example, three images.
In the example system of FIG. 2, mirror sites 22 and 100 are employed in the Japanese network 94. These mirror sites 22 and 100 generally contain copies of content stored in content provider original site 96. Access to a mirror site located within the same network as the local or requesting end user 92 often results in faster access time. Thus, in the example of FIG. 2, when end user 92 in Japan requests data from content provider original site 96, it would generally be much faster if end user 92 was able to access the data from a mirror site 22 or 100 in Japan, rather than accessing the image from content provider original site 96 in the United States and transferring the image across various Internet gateways. Of course, end user 92 must somehow be informed that the desired content can be accessed from a closer (e.g. more quickly accessible) source.
To facilitate this communication, content provider original site 96 maintains a cache table 28, or accesses a cache table 28 maintained by another system, to keep track of the objects that been copied into the various mirror sites. Cache table 28 may contain a list all of the various objects or images that have been copied into other mirror sites, a mapping of the objects and the specific mirror sites that store copies of these objects, and the location of these mirror sites. In addition, cache table 28 may also contain a mapping of partial IP addresses (e.g. the most significant segments of the IP address) to the closest mirror site for those partial IP addresses.)
The xe2x80x9cclosestxe2x80x9d mirror site is the mirror site for which response time is the fastest. Content provider original site 96 uses cache table 28 to locate a closer location from which end user 92 can access the desired content. It should be noted that although end users 92 and mirror sites 22 and 100 may be geographically close, they may be located on completely different networks. Thus, the process of selecting and copying content into a mirror site and informing end user 92 that the desired content can be found in a particular mirror site is not necessarily a simple matter of geography. Content provider original site 96 may determine the closest mirror site using the mapping in cache table 28.
In the system of FIG. 2, when end user 92 first requests access to a web page within content provider original site 96, it is in the form of a xe2x80x9cget index.htmlxe2x80x9d request 26. Content provider original site 96 receives this request and first determines that end user 92 is from Japan because of the IP address information contained in the request. Assume, for purposes of this example, that the requested web page contains obj1, obj2, and obj3. Because content provider original site 96 has access to cache table 28 and also knows that requesting end user 92 is located in Japan, a customized index.html page 48 is sent back to end user 92 containing address information 50 indicating that obj1 is located in mirror site 1 (see reference character 22) in Japan, and that obj2 and obj3 are located in mirror site 2 (see reference character 100), also in Japan.
When end user 92 receives and parses the customized index.html page 48, it determines that there are three embedded objects, obj1, obj2, and obj3. End user 92 then sends one xe2x80x9cget objectxe2x80x9d request for each object. However, instead of requesting the objects from content provider original site 96 in the United States, end user 92 instead requests obj1 from mirror site 1, and requests obj2 and obj3 from mirror site 2. In summary, content provider original site 96 modifies the index.html page sent back to end user 92, and redirects end user 92 to alternate locations where the same object information has been mirrored for faster access. The process is transparent to the end user.
One disadvantage with the embedded object redirection content delivery system of FIG. 2 results from the fact that only objects are mirrored. Thus, although a requested page may contain links to other pages, modified index.htnl page 48 will not contain redirection information for those pages. If a user wants to access these linked pages, a separate page request for each page must be directed to content provider original site 96.
Furthermore, because all pages must be fetched from content provider original site 96, some of the processing capability within content provider original site 96 must be reserved for processing all the initial requests, and preparing and transmitting customized index.html pages 48, which could be large. This added processing may be significant, especially when cache table 28 is very large. In addition, because a modified index.html page 48 must be transmitted by content provider original site 96 for every page request, a full modified index.html page 48 (including the complete text of the page) must pass through the Internet gateways for each page request, which will affect the speed of every page request.
Another disadvantage with the content delivery system of FIG. 2 is related to input/output media delivery. In the system of FIG. 2, although much of the actual transfer of content may occur between end users 92 and mirror sites in locations remote to content provider original site 96, e.g. such as in Japan, content provider original site 96 may continue to have high input/output traffic because it must receive all the initial requests 26 and prepare and transmit a modified index.html page 48 for each request. Thus, even though content may be dispersed into various mirror sites in other locations, crashes may still occur because of excessive input/output traffic associated with content provider original site 96.
FIG. 3 is a block diagram illustrating an example of a typical domain name server (DNS) redirection content delivery system 30.
A brief introduction to DNSs will now be provided. Generally, end users determine the physical address associated with a particular domain name using a network of DNSs. A DNS converts a domain name such as NEC.com, which is easier for humans to remember, into a physical address comprised of a string of numbers, which is appropriate for computer processing, by looking up a table of domain name/IP address mappings. The DNS network is structured like an inverted tree with an international DNS at the top. This international DNS can provide address information to direct a user to other DNSs such as .com or .edu DNSs in the United States, or a .jp DNS in Japan. These DNSs can provide further address information to direct a user to, for example, NEC.com. DNSs may be operated by a particular Internet service provider, or by companies within an intranet.
Thus, for example, if an end user wishes to access NEC.com, the end user first looks up the desired domain name xe2x80x9cNEC.comxe2x80x9d by initiating a request to a local DNS. The local DNS communicates the desired domain name to an international DNS, which then directs the local DNS to a .com DNS in the United States. The .com DNS then directs the local DNS to the server for NEC.com by communicating the physical address of the NEC.com server back to the local DNS.
In the example of FIG. 3, content provider original site 102 is located in the United States. A special-purpose DNS identified as DNS redirection server 34, also located in the United States, is used as a front end for content provider original site 102. In addition, mirror site 2 (see reference character 136) is employed in a Taiwan network 24 and mirror site 1 (see reference character 112) is employed in a Japan network 104. These mirror sites contain a copy of the content stored in the content provider original site 102. Access to a mirror site located within the same network as the requesting end user often results in faster access time. Thus, in the example of FIG. 3, when end user 106 in Taiwan requests content from content provider original site 102, it would generally be much faster if end user 106 was able to access the content from mirror site 2 in Taiwan, rather than accessing the image from content provider original site 102 in the United States and transferring the image across the Internet gateways. Of course, end user 106 must somehow be informed that the desired content can be accessed from a closer source.
FIG. 3 contains a conceptual illustration of how the DNS redirection network performs this function. In FIG. 3, end user 106 in Taiwan, through a local DNS 32, sends a request to DNS redirection server 34 for specific content and asks, in essence, xe2x80x9cwho is the content provider server?xe2x80x9d (See reference character 56.) DNS redirection server 34 then determines the closest mirror site for the requested content (mirror site 2, in this example), and, in essence, sends a message back to local DNS 32 with an answer of xe2x80x9cmirror site 2xe2x80x9d (see reference character 58). Local DNS 32 then communicates xe2x80x9cmirror site 2xe2x80x9d back to end user 106, who would then request the specific content from mirror site 2 (see reference character 82).
To perform this redirection function, DNS redirection server 34 must keep track of the address of each mirror site. A more detailed description of the typical content delivery system of FIG. 3 will now be given with reference to the Japan network 104. When a request for the address of the content provider original site (see reference character 52) is received from a local DNS 108 in Japan network 104, DNS redirection server 34 will determine, from the address information contained within the request, the location of requesting end user 110. DNS redirection server 34 will then identify mirror site 1 (see reference character 112) which is closer to requesting end user 110, based on geographical location and network configuration. DNS redirection server 34 will then communicate the physical address of the identified mirror site 1 back to local DNS 108 (see reference character 54). Thus, local DNS 108 will store the physical address for mirror site 1, rather than the physical address for content provider original site 102. Once local DNS 108 stores the physical address of mirror site 1, future requests for the content provider original site made to local DNS 108 will be immediately directed to the physical address of mirror site 1.
After local DNS 108 stores the physical address for mirror site 1, this address is communicated back to end user 110. End user 110 will then communicate with mirror site 1 (see reference character 60) to obtain the desired content.
Because the DNS redirection content delivery system of FIG. 3 is limited to the redirection of domain name addresses, local DNS 108 only stores the address of mirror site 1, and does not have more specific address information such as the address of other web pages or objects. This is a significant drawback. Specific content must still be located from within mirror site 1. Thus, in the system of FIG. 3, the mirror sites must not only contain a copy of all of the content from content provider original site 102, but also the software such as a database system to search for the desired objects within that content. In contrast, in the system of FIG. 2, only certain objects are stored in the mirror sites, and the actual search for the desired objects is performed in content provider original site 96 in the United States. Thus, in the example system of FIG. 2, mirror sites 22 and 100 act merely as web servers that fetch information.
Referring again to the system of FIG. 3, once local DNS 108 stores the address of mirror site 1, an expiration time begins to count down. During that time, any end user that requests content provider original site 102 through local DNS 108 will be directed to mirror site 1. Once local DNS 108 has stored a physical address for mirror site 1, local DNS 108 will continue sending end users that request content provider original site 102 to that same physical address from that time forward until the expiration time.
Another disadvantage of the system of FIG. 3 is that once a local DNS 108 has stored a physical address for a mirror site, local DNS 108 will send all end users to that same physical address, regardless of the traffic status of the mirror site. If mirror site 112 is overloaded with requests, nothing can be done to remove the load until the physical address for mirror site 112 stored in local DNS 108 has expired.
One variation on the example DNS redirection content delivery system 30 of FIG. 3 (but not shown in FIG. 3) uses a program called a mobile agent to send data and route end users to particular mirror sites in a more controlled fashion. This variation on the system of FIG. 3 interposes some additional functions and intelligence between the end user and the local DNS in the form of one or more processing algorithms. For example, if an end user 110 in Japan requests access to content provider original site 102 for one particular purpose, e.g. to fetch video, that end user 110 might be directed to a mirror site 112 in Japan. However, another end user, also in Japan, may request access to content provider original site 102 for a different purpose, e.g. to purchase airline tickets. This end user might be directed instead to a mirror site in the United States.
A disadvantage with this variation on the system of FIG. 3 is that the various DNSs around the world are operated by different organizations and companies. These entities are generally reluctant to allow an intelligent routing program, not within their control, to reside within their DNSs and redirect traffic to different locations. Such entities generally want to maintain control of routing, and do not want to give up such control. Because of these problems, this variation on the system of FIG. 3 is most applicable when integrated within an organization that owns all of its own DNSs and/or network routers, where there is no division of ownership within routing control.
Therefore, it is an advantage of embodiments of the present invention to provide a system and method for providing content delivery services that allows a requesting end user to be redirected to different mirror sites by transmitting a redirection page back to the end user containing a http protocol meta-tag but no content. The http protocol meta-tag redirects the end user to closely situated mirror sites to access whole pages (the complete text of pages) and objects, minimizing the number of content fetches that must pass through Internet gateways.
It is a further advantage of embodiments of the present invention to provide a system and method for providing content delivery services that allows both pages and objects to be mirrored. Thus, if an end user sends a redirected request to a mirror site for a page containing embedded objects and links to other pages, a whole page will be sent back to the end user that may provide address information for fetching embedded objects and linked pages from mirror sites without having to request the embedded objects and linked pages from the original content provider, thereby minimizing traffic at the original content provider.
It is a further advantage of embodiments of the present invention to provide a system and method for providing content delivery services that allows a selectable portion of a web site, such as the home page, and a selectable number of objects in those pages, to be mirrored.
It is a further advantage of embodiments of the present invention to provide a system and method for providing content delivery services wherein the relationship between the content provider and the content delivery network may be tightly or loosely coupled. In a tightly coupled relationship, the content provider may maintain tables identifying mirrored content and the location of mirror sites which contain that content, maintain a mapping of partial IP addresses to mirror sites closest to those partial IP addresses, and generate modified HTML pages containing redirection information. In addition, the content provider may provide the processing and control for copying content into the mirror sites. In a loose association between the content provider and the content delivery network, the content provider need only maintain a table of which content has been mirrored. When a request for mirrored content is received, the content provider need only direct the requestor to reload the desired content from a special-purpose server, which may then generate modified HTML pages containing redirection information. The ability to have a loosely coupled relationship may be advantageous to smaller content providers, who do not have the resources to establish and maintain a tightly coupled relationship with the content delivery network.
These and other advantages are accomplished according to a content delivery system which includes a content provider original site configured for communicating with end users over at least one communication network. The content delivery system also includes a memory accessible by the content provider original site for identifying which content has been stored in a mirror site. In response to an end user request for particular content, the content provider original site is capable of accessing the memory, determining if the requested content is stored in a mirror site, and communicating a redirection page to the requesting end user containing a meta-tag for redirecting the requesting end user to load the requested content from the mirror site.
These and other objects, features, and advantages of embodiments of the invention will be apparent to those skilled in the art from the following detailed description of embodiments of the invention, when read with the drawings and appended claims.